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FLEXIBLE, EXTENSIBLE, AND PORTABLE TESTING PLATFORM 

TECHNICAL FIELD 

The present invention is related to software, firmware, and hardware 
5 testing and, in particular, to a test execution and test development environment, or 
testing platform, that provides for development of portable test routines, by shielding 
test routines from specific hardware and software interfaces, and that itself can be 
easily ported and enhanced to facilitate testing of many different types of hardware 
and software components while running within a multitude of different computing 
10 environments. 
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03 BACKGROUND OF THE INVENTION 

p| Automated testing is currently a major component of the design, 

jW manufacture, configuration, and installation of hardware, firmware, software, and 

=1;! 15 complex hybrid systems. Although designers, manufacturers, configuration experts, 

JU and installers strive to develop reliable components and systems, it is nearly 

Cm impossible, within commercial economic constraints, to achieve provable correctness 

jij and robustness without an iterative approach in which problems and deficiencies 

P discovered via testing at one stage are corrected in a subsequent stage. A sizable 

20 body of theory and technology related to testing and quality assurance has evolved 
along with the evolution of computers and software technologies. Designers, 
manufacturers, configuration experts, and installers routinely develop sophisticated 
automated testing software for testing software routines and systems, firmware 
routines, hardware components, and complex components comprising combinations 
25 of software, firmware, and hardware. 

Figure 1 abstractly illustrates a common, generic testing environment 
in which an automated test is run. An automated test program 102 executes within a 
computer resource 104, interfacing to, and exchanging data with, an operating 
system 106 that provides system services by interfacing to hardware and firmware 
30 computing components 108 in the computing resource. Hardware and firmware 
components include communications controllers and device drivers that allow the 
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computing resource 104 to interface to, and exchange data with, external entities that 
include user input/output ("I/O") devices 110 such as display terminals and 
keyboards, data storage and retrieval devices 112 including external disk drives, disk 
arrays, and database servers, and external entities 1 14 under test by the automated test 
5 program 102. When the automated test program 102 tests another software 
component, the software component under test 116 may also be executed in 
association with the operating system 106 and underlying hardware and firmware 
components 108. In addition, additional programs 118 may run in association with 
either the program under test 116, in association with the automated test program 102, 
10 or in association with both program under test and with the automated test program. 
Examples of associated programs include database management systems, simulators, 
and specialized drivers. Thus, the environment 100 in which an automated test 
program 102 runs may be quite complex, and may involve numerous 
hardware/hardware, hardware/software, and software/software interfaces. 
15 Great time and effort may be involv^cfin developing test programs. In 

many cases, test programs are developed inX relatively ad hoc manner to include 
dependencies on many different hardware/hardware, hard ware/so ft ware, and 
software/software interfaces. Foi^example, it is quite common to directly embed 
\ operating-system-specific system calls directly within an automated test program. 



20 Embedding system calls dk'ectly within the automated test program may represent an 
efficiency or expediency with regard to developing a test routine for of a particular 
component, but jtfay, in turn, represent a burdensome dependency if the automated 
test program/is attempted to be applied to testing a different type of component or is 
attempted to be ported to execute under a different operating system from that for 
25 whi<^h the automated test program was initially developed. 



discussed above, the automated test program 102 may be quite dependent on the 
interface 120 to the operating system 106. The automated test program 102 may 
additionally depend on a particular interface 122 used to access and exchange data 
30 with a hardware component 1 14 under test. The automated test program may depend 
on a particular user I/O interface 124, including a protocol by which data exchanged 




Many possible interface dependencies are visible in Figure 1. As 



with an external I/O device 110 is interpreted by the automated test program 102. 
The automated test program 102 may also depend on high-level interfaces, not shown 
in Figure 1, between the automated test program 102 and a program under test 1 16 or 
an associated, concurrently running program 118. 
5 Embedded hardware/hardware, hardware/software, and 

software/software interface dependencies in an automated test program may inhibit or 
prevent using the automated test program to test components other than the specific 
components for which it is designed, and may inhibit execution of the automated test 
program in environments different from the environment 100 for which the 
10 automated test program is developed. A non-modular design of the automated test 
y program 102 may inhibit or prevent the automated test program from being enhanced 

61 or extended to execute different types of tests or to execute tests with different 

r\ frequencies, ordering, and configuration. 

jf{ Attempts have been made to generalize automated test programs, so 

*B 15 that the automated test programs can be enhanced, or extended, ported to different 

%i hardware and operating system environments, and employed to test a variety of 

different software or hardware components. However, currently available automated 
fy test programs and automated test program environments have not achieved a desirable 

p level of enhancibility, portability, and applicability. For this reason, designers, 

20 manufacturers, configuration experts, and installers have recognized the need for a 
modular and easily enhanced, portable, adaptable, and generalized testing platform to 
serve as an environment for running automated test programs. 

SUMMARY OF THE INVENTION 

25 One embodiment of the present invention is en extensible, adaptable, 

and portable testing platform comprising a test execution engine and one or more 
automated test routines, the test execution engine and automated test routines 
shielded from hardware/hardware, hardware/software, and software/software 
dependencies by abstract functional components, specific instances of which are 

30 adapted to interface to particular hardware and software resources and components in 
the testing environment. The test execution engine includes a relatively small internal 
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execution loop that is free from dependencies on the computing resource within 
which the test execution engine runs, and is free from dependencies on particular 
automated test programs, user I/O interfaces, and interfaces to external computing 
resources and components. Likewise, the one or more automated test routines can be 
5 written without dependencies on the computing resource in which they are run or on 
user I/O interfaces, component interfaces, and other such dependencies. The testing 
platform is therefore easily enhanced, easily adapted to test entities unforeseen at the 
time that the testing platform is developed, and is easily transported to a myriad of 
different computing resource environments. 

10 

BRIEF DESCRIPTION OF THE DRAWINGS 

Figure 1 abstractly illustrates a common, generic testing environment 

in which an automated test is run. 

Figure 2 illustrates a testing platform that incorporates aspects of the 

15 present invention. 

Figure 3 shows a high-level class structure for functional components 
surrounding the test execution engine of one implementation of a testing platform that 
incorporates aspects of the present invention. 

Figure 4 shows a high-level class structure for functional components 
20 surrounding a test routine in one implementation of a testing platform that 
incorporates aspects of the present invention. 

DETAILED DESCRIPTION OF THE INVENTION 

One embodiment of the present invention is a dependency-shielded 

25 test execution engine and one or more dependency-shielded test routines that, 
together with a number of functional components adapted to particular 
hardware/hardware, hardware/software, and software/software interfaces, comprises 
an easily modifiable, adaptable, and portable testing platform. Any full 
implementation of the testing platform that represents one embodiment of the present 

30 invention greatly exceeds the scope of a concise description but, fortunately, the bulk 
of implementation details are straightforward and well within the implementation 
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ability of one skilled in the art of test platform development. The present invention 
relates to ^ core architecture and a number of core concepts that enable 
implementation of an almost limitless number of easily modifiable, adaptable, and 
portable testing platform implementations. One embodiment of the present invention 
5 is described below in three subsections: (1) a high-level overview of a testing 
platform that incorporates the present invention; (2) a high-level, partial description 
of the modular organization of, and class declarations contained in, a real-life 
implementation of a testing platform that incorporates aspects of the present 
invention; and (3) an actual C++ implementation of the test execution engine of a 
10 testing platform implementation that incorporates aspects of the present invention. 

Overview 

Figure 2 illustrates a testing platform that incorporates aspects of the 
present invention. The computing resource and environment within which the testing 

15 platform resides include various operating-system-provided functionalities 202, data 
storage and output devices 204-205, user I/O devices 206-207, and, optionally, a 
hardware component 208 that is tested by a test routine running in association with 
the testing platform. A single test routine 210 is shown in Figure 2. A testing 
platform incorporating the present invention may concurrently or sequentially run a 

20 large number of different testing routines, the testing routines running in an 
asynchronous or synchronous manner. 

The testing platform includes a core test execution engine 212 that 
includes a central execution loop that continuously executes in order to run one or 
more test routines according to various user-input and programmed parameters. Both 

25 the test routine 210 and the test execution engine 212 are shielded by additional 
testing platform components from dependencies on the computing resources and 
external entities 202, 204-205, 206-207, and 208 within the environment in which the 
testing platform runs. The test execution engine 212 interfaces to data output and 
data storage devices 204-205 via a result handler component 214. The test execution 

30 engine interfaces to operating-system-provided functionality 202 via various 
components diagrammed together in Figure 2 as a generalized operating system 
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adaptor 216. The test execution engine interfaces to user I/O devices 206-207 via a 
user I/O component 218. The test execution engine interacts with the test routine via 
a mode component 220, a sequencer component 222, and a test executor 
component 224. The test routine 210 interfaces with the result handler component 
5 via a first test link component 226 and interfaces with the operating system adaptor, 
user I/O handler, and a communications interface 228 via a second test link 
component 230. The communications interface component 228 serves to interface 
the test routine 210 with a hardware component 208 tested by the test routine. 

The test execution engine 212 is a centralized, core component of the 
10 testing platform. It includes the fundamental execution loop of the testing platform, 
~, and is also the location of the instantiation of class objects that represent many of the 

remaining functional components of the testing platform. 
HI The mode component 220 is the main interface between the test 

H execution engine 212 and all other components and resources of the testing platform 

w 

Hi 15 and its computing environment. A mode defines the semantic meaning of user input 

I" to the testing platform, including input that causes the current mode to terminate and 

5 a new mode to assume its place. A testing platform may include many different 

oi 

H instantiations of the mode class and derived mode classes or, in other words, many 

jif different mode objects. Each mode object defines the overall behavior of, and user 

H 20 interface to, the testing platform. The various mode classes may be related to one 

another through inheritance. For example, in one embodiment, a number of 
derivative mode classes are derived from an online-mode class, and a number of 
additional derivative mode classes are derived from an off-line-mode class. 

The online-mode class provides an ability to update test parameters 
25 and view test description and result information. Test selection is via a cursor that 
moves up and down through a current page of a number of pages that describe, to a 
user, a suite of tests. A sequential-mode class derived from the online-mode class 
allows tests to be run and completed in sequence. A random-mode class derived from 
the online-mode class also launches test routines sequentially, but randomly varies the 
30 parameters so that, on each iteration, the test routine is called with slightly different 
parameters. Tests are run asynchronously under random mode. The off-line-mode 
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class allows for complete customization of the testing platform so that, for example, 
an interactive hardware testing and debugging environment can be implemented. 

The sequencer component 222 is, in one embodiment, implemented as 
a test sequencer class. As with the mode class, a number of derived test sequencer 
5 classes may be related via an inheritance tree to a parent test sequencer class. Some 
testing sequencer classes provide hard-coded test execution sequencing. For 
example, a suite of tests may be run in sequential order when one derived test 
sequencer class is employed, and other types of execution ordering and execution 
behaviors are offered by other derived test sequencer classes. For example, in another 

10 derived test sequencer class, selected tests may be continuously run until stopped by a 
user or by a detected error, the continuous execution ordered in various different 
~ ways. Other derived test sequencer classes allow for specialized test sequence 
behavior via programming. Thus, almost any test sequencing behavior can be 
provided by the testing platform by employing either a hard-coded derived test 

1 5 sequencer class or by employing a specialized, programmed test sequencer class. 

The test executor component 224 provides an abstract and generalized 
test interface to the test sequencer component 222. The test sequencer 
component 222 interacts with test routines via a numerical test identifier provided by 
the test executor component 224. Thus, the test sequencer component 222 is shielded 

20 from interface and implementation details associated with any particular type of test 
routine. The test executor component 224 allows the test sequencer component 222 
to interface to a variety of different types of test routines, including embedded tests 
that are linked together with the executable testing platform, separate executable test 
routines that are not linked with the testing platform executable but that are 

25 executable on the computing resources that support running of the testing platform, 
and external test routines that run outside the computing environment of the testing 
platform and that are accessed via hardware or software interfaces. Thus, the test 
execution engine 212 can execute a great many of different types of test routines 
adapted by the test executor component 224 to a common interface with the test 

30 sequencer component 222. 
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The results handler component 214 provides a generic interface for 
outputting test results to various external media, including printers 204, and hard- 
disk-based files and databases 205. Additional results output sinks can be easily 
added by including specifically adapted results handler components. The user I/O 
5 component 218 provides display handling and user input functionality. Thus, the 
testing platform can support an almost unlimited number of different types of user 
interaction, including user interaction via graphical user interfaces, consoles, and 
other types of user information output devices, and can select and coalesce user input 
from a variety of different user input devices, such as computer keyboards, touch 

10 screens, face recognition systems, and other such input devices. The operating- 
system-provided services interface component 216 includes various classes that 
provide operating-system-specific memory management 230 and timer 232 
functionalities, as well as other types of operating-system-provided services. 
Communications interface components 228 provide interfaces to external entities 

15 accessed by the test routine 210, including external entities tested by the test routine. 
The test link components 226 and 230 provide to the test routine 210 a generalized 
interface to the testing platform, and to the functional components that make up the 
testing platform, shielding the test routine from testing platform details and enabling a 
test routine to be written easily and without dependencies on computing resource 

20 environments and testing platform internal and external interfaces. 

The highly modular and onion-like layers of shielding provided by the 
functional components of the testing platform allow for the high levels of 
enhancability, adaptability, and portability of both the testing platform and of test 
routines developed to test various hardware and software components. The testing 

25 platform, for example, can be ported to almost any computing resource environment, 
including to many different operating systems and computer platforms, without the 
need to modify the internal execution loop within the test execution engine, nor 
functional components such as the test sequencer. Similarly, different test sequencing 
paradigms can be implemented in derived test sequencer classes without requiring 

30 notification to other functional components of the testing platform, including the test 
executor component 224 and the mode component 220. The basic functional 



components are thus implemented within distinct classes that interact through defined 
interfaces. 
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High-Level Class Organization Of One Embodiment Of A Testing Platform That 

Incorporates Aspects Of The Present Invention 
Figure 3 shows a high-level class structure for functional components 
5 surrounding the test execution engine of one implementation of a testing platform that 
incorporates aspects of the present invention. Figure 4 shows a high-level class 
structure for functional components surrounding a test routine in one implementation 
of a testing platform that incorporates aspects of the present invention. 

In Figures 3 and 4, parent classes are shown as squares or rectangles 
10 with heavy borders, such as parent class 302. Derived classes that inherit from a 
parent class are shown as squares or rectangles with fine borders, such as derived 
class 304 in Figure 3, with an arrow, such as arrow 306 in Figure 3, interconnecting 
the derived class with the parent class and directed from the derived class to the 
parent class. A class that contains a number of instances of, or pointers to, another 
15 class is known as an "aggregate class," such as, for example, aggregate class 308 in 
Figure 3, and the "contains" relationship is indicated by a line, such as line 310 in 
Figure 3, from the container class terminating with a small diamond-shaped object, 
such as diamond-shaped object 3 12 in Figure 3, adjacent to the contained class. 
Interrelationships between classes shown in Figures 3 and 4 are indicated by circles 
20 with an identifying label, such as circles 402 in Figure 4 and 314 in Figure 3, both 
containing the common label "A." Thus, the user I/O interface class 316 is an 
aggregate class containing a number of references or instances of the communication 

interface class 404. 

Many of these classes shown in Figures 3-4 have been described 

25 above, with reference to Figure 2. Those descriptions will not be repeated, in the 
interest of brevity. The UUT Class 318 contains specific textual numerical 
information about the units under test that may be included in output reports. The test 
object class 320 contains descriptive information about a test routine. The project 
configuration class 406 contains information about a non-volatile data storage entity. 

30 The test class 408 is an aggregate class that describes a number of different sets of 
test routines, incorporated within test links that allow the test routines to access user 
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I/O components, operating-system-providing functionality components, and other 
functional components of the testing platform. Figures 3 and 4 are provided to assist 
an attentive reader in understanding the C++ test execution engine implementation 
provided in the next subsection. 



C++ Test Execution Engine Implementation 
The following is an actual C++ implementation of the test execution 
engine component of a testing platform that incorporates concepts of the present 
invention. This code will be first provided below, in total, and will then be described 

10 and annotated in the text that follows: 

1 #define PRINT_ON1 

2 #include ,, Global_defs.h n 

3 #include <iostream.h> 

4 #include "win_results_to_dax.h" 

5 #include "proj_configuration.h" 

6 #include "tlink_to_embedded_results.h ,, 

7 #include "programmable_sequencer.h" 

8 #include "tst_mode.h M 

9 #include "onlin^seqjnode.h" 

10 include "mode.def.h" 

1 1 #include M Win_com.h n 

12 include M tst_mode_defs.h u 

13 #include "ontine^ode^defs.h" 

14 #include "test_suite_template.h n 

15 #include "test_vec_defs.h M 

16 #include "test_object.h" 

17 #include "test_objects_map.h" 

1 8 #include "online_std_sequencer.h M 

19 #include "embedded_executor.h M 

20 #include M windows_executor.h ,, 

21 #include "test_executor_defs.h" 

22 #include ,, tlink_embedded_use^io.h ,, 

23 #include ,, UUT_information.h M 

24 #include M UUT_defs.h" 

25 #include "results_handler.h M 

26 #include "PC_Timer.h M 

27 #include "winJcl_tk_userio.h" 

28 #include "winnt_scsi_com.h M 

29 #include n File_print.h" 

30 #include "windows.h" 

31 #include <stdio.h> 

32 void exec_sleep(unsigned long time); 

33 void error_exit(char* msg); 

34 bool parse_input_params(int argc, char** argv, struct MAI N_l N PUT_PARAMS & 

35 main_input_params); 
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35 



40 



45 



36 sys_mode_ptr mode_ptr; 



t 
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37 PCJTimer pcjimer; 

38 Test_Objects_Map test_objects_map; 



5 39 UUTJnformation uut_object((Timer*)&pc_timer); 

40 uut_data_union uut_data = { 

41 "ManufacturingFunctionalTest", //Test name 

42 "123456", // Part number 

1 0 43 "", // Serial number - user entered 

44 m \ II Host/Computer name (retrieved) 

45 m \ II Test Slot - if applicable 

46 "1.0", // Test Version - if applicable 

47 "1 23456", // Product - if applicable 
1 5 48 "123456", // Model - if applicable 

49 ul, ( // Sub Family - if applicable 

50 m \ II Standard Operating System 

51 m \ II Operator name/number 

52 0 // start tick 
20 53 }; 

54 char* header = "MANUFACTURING FUNCTIONAL TEST RESULTS"; 

55 int interval = 25; 

25 56 Proj_Configuration proj_config("DAX_fileJoc.txt"); 

57 Proj_Configuration test_file("test_suite_file.txt M ); 

58 win_com win_rs232( ,m ); 

59 dword win_rs232_init[7] = 

30 60 {CBRJ38400, FALSE, TRUE, TRUE, NOPARITY, ONESTOPBIT, 

FALSE}; 

61 WinNT_SCSI_Com winnt_scsLcomm; 

35 62 //UserJO_rs232 user_io_term(win_rs232); 

63 //UserlO_Win_Console user_io_win; 

64 Win_Tcl_TkJJserJO winJcLgui(0, WIN_TCL_FILE); 
65 

66 User JO* user_io_array[] ~ 

40 67 { 

68 &win_tcl_gui, (UserJO*)0 

69 // &win_tcLgui, userjojerm, &user_io_win, (User_IO*)0 

70 }; 

45 71 Win_Results_to_DAX win_dax_results(uut_object, mode_ptr, user_io_array, 

72 &proj_config); 

73 Results_Writer* res_dbase_array[] = 

74 { 

75 &win_dax_results, (Results_Writer*)0 
50 76 }; 
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77 /* 

78 Win_Results_to_File win_file_results(uut_object, mode„ptr, user_io_array t 

79 "win_results_file.txt"); 

80 Results_Writer* res_file_array[] = 

81 { 

82 &win_file_results, (Results_Writer*)0 
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83 }; 

84 Win_Resutts_to_Printer win_printer_resutts(uut_object, mode_ptr, 

85 userjo_array, 

5 86 "HP DeskJet 720C v10.3"); 

87 Results_Writer* res_printer_array[] = 

88 { 

89 &win_printer_results, (Results_Writer*)0 

90 }; 
10 91 */ 
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92 //ResultsJHandler results_handler(test_objects_map, res_file_array, 

93 // res_printer_array, res_dbase_array); 

94 ResultsJHandler results_handler(test_objects_map, 0, 0, res_dbase_array); 



95 const int userjojndex = 0; 

96 Tlink_em bedded jjserio tlink_userio(mode_ptr, user_io_array, userjojndex) 
20 97 TlinkJo_Embedded_Results tlink_results(results_handler); 

98 file_print file_printjink(); 

99 Test_Suite_Template example_test_suite(pc_timer, win_rs232, tlinkjjserio, 
25 100 tlink_results, test_file, 

101 wi n nt_scs i_c om m ) ; 

102 Test_Vector* template_array[] = 

103 { 

30 104 (Test_Vector*)&exampleJest_suite, (Test_Vector*)0 

105 // (Test_Vector*)&simpieJemplate, (Test_Vector*)0 

106 }; 



107 embedded_executor embed_executor(template_array); 

108 const int win_io_dex = 0; 

109 windows_executor win^executo^userjo^array, win_io_dex, mode_ptr, 

110 resultsjiandler, DEF_WIN32_TEST_FILE); 



1 1 1 Test_Executor* executor_array[] = 

112 { 

1 1 3 &embed_executor, (Test_Executor*)0 

114 // embedded executor and windows O/S executable 
45 115 // &embed_executoi\ &win_executor, (Test_Executor*)0 

116 }; 

1 17 on1ine_std_sequencer std_seq_sequencer(uut_object, test_objects_map, 

118 executor_array, results_handler); 

50 119 Programmable_Sequencer programmable_sequencer(uut_object, test_objects_map, 

120 executor_array, resultsjiandler, 

121 "sequencer.txt"); 

122 tstjnode my_testjriode(DEFJ<EY_FILE); 

55 123 online_seq„mode onLseq_mode(test_objects_map, &std_seq_sequencer, 

124 resultsjiandler, uut_object, 

125 SEQ_KEY_FILE, 

126 &programmable_sequencer); 
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127 sys_mode_ptr mode_array[] = 

128 { 

129 &my_test_mode t &onLseq_mode, (sys_mode_ptr)0 

130 }; 

5 

131 struct COMM_STRUCT 

132 { 

133 comjf* com_ptr; 

134 void* init_param; 
10 135 }; 

136 struct COMM_STRUCT comm_array[] = 

137 { 

1 38 {&win_rs232, win_rs232 jnit}, 

139 {(com_if*)0, (void*)0} 
15 140 }; 
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141 #define OPTIONS 8 

142 struct MAINJNPUT_PARAMS 

143 { 

20 144 bool replay; 

145 charuut[21]; 

146 long unsigned int virtuaLport; 

147 char filename[81]; 

148 char test_mapfile[81]; 
25 149 char pro_seq_file[81]; 

1 50 char config_file[81]; 

151 char tcl_tk_guLfile[81]; 

152 }; 
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// start auto replay 

// UUT Part# 

// virtual User I/O port 

// auto replay of keys file 

//test mapping file 

// programmable sequencer file 

// project specific config file 

// Tck/Tk GUI Script file 



30 153 struct MAIN JNPUT_PARAMS main_input_params = 



154 {FALSE, ,m , 0, 



MM MM Hit 11M MM 



" "\ ■ 
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155 int main(int argc, char** argv) 

156 { 

157 int ret_code; 

1 58 int comm jf_dex; 

1 59 int user_io_dex; 

160 int res_dex; 

161 int mode_dex; 

1 62 key_def key; 

163 bool param_ret; 
164 



// method return code 
// Comm l/F Object index 
// User I/O Object index 
// Results writer array index 
// Mode Object index 
// command key 
// bool return value 



char err_msg[DISP_TEXT_COLS+1]; // error message 



45 



50 
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165 PRINT1 ("Entered main() M ); 

166 mode_ptr = mode_array[ONLINE_MODE_SEQ]; 

167 ret_code = uut_object.set_uut_information(&uut_data); 

168 if (ret_code != NO_ERR) 

169 error_exit( H ERROR - Writing to the UUT Object\n H ); 

170 param_ret= parse_input_params(argc, argv, mainjnput_params); 

171 if (param_ret) 

172 { 

173 PRINT2( M replay = mainjnput_params. replay); 

174 PRINT2("filename = mainjnput^params.filename); 

175 PRINT2("Part# = ", mainjnput_params.uut); 
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176 PRINT2("port = main_input_params.virtual_port); 

177 PRINT2("test mapfile = main_input_params.test_mapfile); 

178 PRINT2("prog. sequencer file = main_input_params.pro_seq_file); 

179 PRINT2( M configuration file = ", mainJnputjDarams.config_file); 
5 180 } 

181 if (param_ret) 

1 82 { // at least one parameter passed 
10 183 ret_code = NO_ERR; 

1 84 // check for auto replay file 

1 85 if (strcmp(mainjnput_params.filename, "")) 

186 { 

15 1 87 ret_code = mode_ptr->set_filename(main Jnput_params.filename); 

188 } 
189 

1 90 // check for project configuration file 

191 // will change the DAX location file name 

20 192 if (strcmp(mainjnput_params.config_file, "")) 

193 { 

194 ret_code |= proj_config.set_config_name 

1 95 (main_input_params.config_file); 

196 } 

25 

1 97 // check for Programmable Sequencer file 

1 98 if (strcmp(mainJnput_params.pro_seq_file, "")) 

199. { 

200 ret_code |= programmable_sequencer.set_seq_file 

30 201 (mainjnput_params.pro_seq_file); 

202 ret_code |= programmable_sequencer.create_sequence_table(); 

203 } 

204 if (strcmp(main_input_params.test_mapfile/ m )) 

35 205 { 

206 ret_code |= Test_Executor :: 

207 set_mapfile(main_input_params.test__mapfile); 

208 } 

40 209 if (strcmp(main_input_params.tcLtk_gui_file t "")) 

210 ret_code |= 

21 1 win_tcl_gui.set_guLfilename 

212 (main_input_params.tcLtk_gui_file); 

45 213 if (main_input_params. replay) 

214 { 

215 ret_code |= mode_ptr->get_saved_keys_fromJlle(); 

216 ret_code |= mode_ptr->repiay_saved_keys(); 

217 ret_code |= mode_ptr-> 
50 218 display_msgjine 

219 (L_STATUS_STRING, REPLAYED, user_io_array); 

220 } 
221 

222 if(strcmp(main_input_params.uut, "")) 

55 223 { 

224 PRINT2("UUT # Passed = mainjnput_params.uut); 

225 ret_code |= uut_object.get_uutjnformation(uut_data); 

226 strcpy(uut_data.p_uut_data.partnum, main_input_params.uut); 
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} 



ret_code |= uut_object.set_uut_information(&uut_data); 

} 

if (main_input_params.virtual_port != 0) 
{ 

PRINT2("Virtual Port # Passed = ", 

m ai n_in put_pa ram s . vi rtua l_port) ; 

} 

if (ret_code != NO_ERR) 

error_exit("ERROR with Input Parameter(s) 
handling routine(s)\n"); 



comm_if_dex = 0; 

PRINT1 ("INITIALIZING THE USER l/O's, 

PLEASE WAIT, IT TAKES A FEW SECONDS...."); 
user_io__dex = 0; 

while ( user_io_array[userjo_dex] != (User_IO*)0 ) 
{ 

ret_code = 

user_io_array[useMo_dex++]->initialize_userjo((void*)0); 

if (ret_code != NO_ERR) 

error_exit("ERROR - Initializing User I/O object\n M ); 

} 

PRINT1("User I/O Objects Initialized okay"); 
mode_dex = 0; 

while ( mode_array[mode_dex] != (sys_mode_ptr)0 ) 

^ ret_code = mode_arraytmode_dex]->initialize_mode(user_io - array); 
if (ret„code != NO_ERR) 

error_exit("ERROR - Initializing System Mode objecftn"); 

// pass the UUT object to the System Mode 
mode_array[mode_dex]->setJJUT_object(&uut_object); 

++mode_dex; 

} 

PRINT1("Mode Objects Initialized okay"); 
res_dex = 0; 

while ( res_dbase_array[res_dex] != (Results_Writer*)0 ) 
{ // init the results to dbase objects and send header info 

ret_code = res_dbase_array[res_dex]->send_header(header t interval); 
ret_code = res_dbase_array[res_dex++]->open((void*)0); 
if (ret_code != NO_ERR) 

error_exit("ERROR - Initializing Dbase Results 
Writer Object\n"); 

} 

PRINT1 ("Dbase Result Writer Objects Initialized okay"); 
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273 /**** 

274 res_dex = 0; 

275 while (res_file_array[res_dex] != (Results_Writer*)0 ) 

276 { // init the results to file objects and send header info 

277 ret_code = res_file_array[res_dex]->send_header(header, interval); 

278 ret_code = res_file_array[res_dex++]->open((void*)0); 

279 if (ret_code != NO_ERR) 

280 error_exit("ERROR - Initializing File 

281 Results Writer ObjecUn"); 

282 } 

283 PRINT1 ("File Result Writer Objects Initialized okay' 1 ); 



284 res_dex = 0; 
15 285 while (res_printer_array[res_dex] != (Results_Writer*)0 ) 

286 { 

287 ret_code = res_printer_array[res_dex]->send_header(header, 0); 

288 ret_code = res_printer_array[res_dex++]->open((void*)0); 

289 if (ret__code != NO_ERR) 

20 290 error_exit( M ERROR - Initializing Printer 

291 ~ Results Writer Object\n"); 

292 } 

293 PRINT1("Printer Result Writer Objects Initialized okay n ); 
25 294 *****/ 

295 ret_code = mode_ptr->display_main_screen(user_io_array); 

30 296 user_io_dex = 0; 

297 while (TRUE) 

298 { 

299 if (user_io_array[user_io_dex] == (User_IO*)0) 

300 user_io_dex = 0; 

35 

301 ret_code = mode_ptr->get_command(userjo_array[user_io_dex], key); 
302 

303 if (ret_code == NO_ERR) 

304 { 

40 305 ret_code = mode_ptr->verify_general_cmd(key); 

306 if (ret_code == NO_ERR) 

307 { 

308 ret_code = 

309 mode_ptr->execute_generaLcmd(user_io_array, 
45 310 user_io_dex, key, 

311 mode_ptr, 

312 mode_array); 
31 3 

314 if (ret_code == EXIT_EXEC) 

50 315 { 

316 results_handler.signaL_end_results(); 

317 break; 

318 } 

319 } 
55 320 else 

321 { 

322 ret_code = mode_ptr->verify_custom_cmd(key); 

323 if (ret_code == NO_ERR) 
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324 { 

325 ret_code = mode_ptr->execute_custom_cmd( 

326 user_io_array, userjo_dex, key); 

327 } 

5 328 else 

329 { 

330 sprintf(err_msg, "Key/Command '%c' NOT supported", 

331 key); 

332 ret_code = 

1 0 333 mode_ptr->displayjrisgJine(L_ERROR_STRING, 

334 err_msg, 

335 user_io_array); 

336 } 

337 } 
1 5 338 } 

339 ret_code = mode_ptr->execute_next_mode_op(user_io„array); 

340 user_io_dex++; 

341 exec_sleep(1 00); 
20 342 } 

343 return(O); 



344 } 

The above C++ test execution engine code is taken from an actual 
25 testing platform implementation, and thus includes many details tangential to the 
present invention. However, for the sake of completeness, the entire module is 
provided, above. In the following discussion, the module is described with particular 
emphasis on those portions of the code that relate to aspects of the present invention 
described in the previous overview and class-description subsections. 
30 Lines 1-31 include C++ include directives that import a number of 

header files that contain class definitions for many of the classes described above, in 
the previous subsections. The broad functionality provided by those classes, as 
described above, is pertinent to a description of the current invention, but 
implementation details and specifics of the class definitions are outside the scope of 

35 the present discussion. 

On lines 32-35, three C-function prototypes are provided. Their 

effects will be described later, at the point that they are called. 

Next, an extended section of global object instantiations begins on line 
36.. The global object "mode_ptr," instantiated on line 36, is a global reference to the 
40 currently active mode of the testing platform, corresponding to the mode 
component 220 in Figure 2. The global object "pcjimer," instantiated on line 37, is a 
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system timer corresponding to timer 232 in Figure 2 packaged within one of the many 
functional components represented in aggregate by functional component 216 in 
Figure 2. The global "test_objects_map," instantiated on line 38, is a container object 
that contains a descriptive test object for each test run by the testing platform. The 
5 global object "uutobject," instantiated on line 39, contains descriptive information 
concerning the component under test by the testing platform, including the descriptive 
information within the union "uut_data," instantiated on lines 40-53. The global 
character string referenced by the pointer "header," declared on line 54, is a header 
string output to result files after output of each group of test results, where the size of 
10 a group of test results is defined by the global "interval," declared on line 55. The 

p globals "proj_config" and "test_file," instantiated on lines 56 and 57, specify disk 

files used for non-volatile storage of database data and test data, respectively. Several 

SI communications interface objects, corresponding to instances of the functional 

jTs component 228 in Figure 2, are instantiated and initialized on lines 58-61. Several 

15 user I/O components, representing instances of functional component 218 in Figure 2, 

a™" are instantiated on lines 62-64, with several instantiations commented out in the 

current implementation because they are not used. Global array "user_io_array" is 

u § 

h* declared and initialized on lines 66-70. This array contains pointers to user I/O 

objects that interface the testing platform to various user I/O interfaces. 

'H 8 20 On lines 71-91, a number of Results_Writer objects are instantiated for 

handling results outputs to databases, disk files, and printers. Certain of these 
instantiations are commented out, because the functionality is not required in the 
current implementation. The commented-out instantiations have been left in the code 
in order to demonstrate how such Results_Writer objects would be included, if 
25 needed. On line 94, the global "results_handler" is instantiated. This 
Results_Handler instance corresponds to the functional component 214 in Figure 2, 
and is responsible for interfacing the testing platform to all data output interfaces, 
including interfaces to databases, files, and printers. On line 95, the global 
"user_io_index" is initialized to zero. 
30 The Tlink objects "tlink_results" and "tlink_userio," instantiated 

above on lines 97 and 96, respectively, correspond to the functional components 226 
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and 230 in Figure 2. These Tlink objects are used by test routines to interface to 
testing platform functional components, including the communications interface 
components, user I/O components, and result handler components. The Tlink object 
"tlink_userio" is passed the global "user_io_index" to indicate that a test routine 
5 using this Tlink object uses the first user I/O object in the array "user_io_array" for 
input. The global function "file_print_link()," declare on line 98, is an event log that 
testing platform developers can use to log various developer-defined events during 
debugging and analysis of the testing platform. 

The global "example_test_suite," instantiated on lines 99-101, 

10 represents a suite, or set, or test routines linked to the testing platform. The array 
"template_array," declared and initialized on lines 102-106, include references to the 
suites, or sets, of test routines that will be run by the testing platform. Grouping of 
test routines into suites allows the testing platform to supply different ordering and 
execution paradigms to the test routines of each different test suite. 

15 Test_executor objects are instantiated on lines 107-1 10. Each 

test_executor object corresponds to functional component 224 in Figure 2. The array 
"executor_array," declared and initialized on lines 111-116, includes references to the 
various different test_executor objects instantiated for the testing platform. As 
described above, a test_executor object interfaces the testing platform to test routines 

20 of particular types, the types including embedded tests that are linked together with a 
testing platform code, test routines that are separate executable programs that run on 
the computing resources environment in which the testing platform runs, and external 
test routines that may run on remote computers or that may represent external 

hardware devices under test. 

25 A number of different sequencer objects, each corresponding to 

functional component 222 in Figure 2, are instantiated on lines 117-121. These 
objects isolate test-sequencing details from the mode objects, and allow interchange 
of various different sequencings of test routine execution. Two sequencer objects are 
instantiated: a standard sequencer is instantiated on lines 117-118 and a 

30 programmable sequencer instantiated on lines 119-121. 
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A number of different system mode objects, each responding to 
functional component 220 in Figure 2, are instantiated on lines 122-126. References 
to these system mode objects are then included in the array "mode_array," declared 
and initialized on lines 127-130. This array contains references to the different 
5 instantiated mode objects that may define operation and behavior of the testing 
platform, as described above. 

On lines 131-140, initialization parameters for communications 
interface objects are stored in global structures. Values of input parameters to the test 
execution engine are described by the structure "MAIN JNPUTP ARAMS" declared 

10 on lines 142-152. An instance of this structure is initialized to default values on 
lines 153-154, and is later passed to a parsing routine that extracts values from 
arguments supplied to the test execution engine "main" routine. 

Finally, the test execution engine is provided on lines 155-344. On 
lines 157-164, local variables for the test execution engine are declared. The local 

15 "user_io_dex," declared on line 159, is used by the test execution engine to 
continuously iterate through the user I/O objects contained in the array 
"user_io_array," declared above on line 66. On line 166, the global "mode_pointer" 
is initialized to point to the online mode object contained within the array 
"mode_array." On line 167, the global "uut_object" is initialized using data stored in 

20 the global union "uut_data." On line 170, the C routine "parse Jnput_params" is 
called to parse the parameters supplied to the test execution engine. A long section, 
including lines 171-238, involves checking and verifying the parsed parameters and 
taking various actions depending on the parameter state. For example, actions may be 
taken to elicit input of parameters initially input in an incorrect format or with values 

25 outside reasonable or expected ranges. On lines 239-295, various global objects 
corresponding to functional components of the testing platform, described with 
reference to Figure 2, are initialized. 

The internal execution loop for the test execution engine is provided 
on lines 296-344. This short section of C++ code represents the basic, core event 

30 handling loop within the testing platform. The fact that the test execution engine can 
be so concisely specified in C++ is a testament to the effectiveness of the dependency 
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shielding provided by the functional components surrounding the test execution 
engine. 

On line 296, the local index "user_io_dex" is initialized to zero. Then, 
the test execution engine continuously operates, within the infinite while-loop of 
5 lines 297-342, until interrupted by a termination event. The whileAoop continuously 
traverses the array "user_io_array" to check each user I/O object for input to the 
testing platform. After each iteration of the whileAoop, the index "user_io_dex" is 
incremented on line 340. If the index is incremented past the end of the I/O objects 
within the array "user_io_array," as detected on line 299, the index "userjodex" is 
10 reinitialized to reference the first user I/O object with the array on line 300. On 
O line 301, the mode member function "get_command" is used to interpret any input, 

available from the user I/O object currently indexed by index "user_io_dex," as a 
j*I command, and that command is stored in the local variable "key." If the user I/O 

hj object had input interpretable by the mode object referenced by the global 

15 "mode_ptr," as detected by the test execution engine on line 303, then, on line 305, 
^ the mode member function "verify_general_cmd" is called to determine whether the 

fn input command is a general command. If so, as detected by the test execution engine 

J! on line 306, the mode member function "execute_general_cmd" is called, on line 309, 

O to carry out the general command. Note that a general command may result in a 

20 change of active modes, necessitating passing of the pointer "mode_ptr" to member 
function "execute_general_cmd." If, as a result of execution of the general command, 
a termination event occurs, forcing termination of the test execution engine, as 
detected on line 314 by the test execution engine, an indication of the termination is 
output via the results handler on line 316 and the while-loop is terminated on 
25 line 317. If the command entered via the user I/O object is a custom command, as 
determine on lines 322-323, then the mode member function "execute__custom_cmd" 
is called on line 325 to process the custom command. If the input from the user I/O 
object cannot be interpreted by the mode object as either a general command or a 
custom command, then an error message is displayed to the user on lines 330-335. 
30 On line 339, the test execution engine executes a next system-mode operation by 
calling the mode member function "execute_next_mode_op." This next system-mode 
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operation depends on the currently active system mode, or, in other words, on the 
contents of global reference "mode_ptr." Different types of system modes provide 
different functionalities via system-mode operations. After incrementing the next 
"user_io_dex" on line 340, the test execution engine calls the C function 
5 "exec_sleep" on line 341 in order to pause to allow other computational activity 
concurrently executing with the testing platform to run within the computing resource 
environment. The call to exec_sleep is therefore, essentially, a yield operation. 
When the test execution engine loop is terminated by the call to the C++ directive 
break on line 317, the test execution engine completes via the statement "return(O)" 
10 on line 343. 

Although the present invention has been described in terms of a 
particular embodiment, it is not intended that the invention be limited to this 
embodiment. Modifications within the spirit of the invention will be apparent to 
those skilled in the art. For example, an almost limitless number of different 

1 5 implementations of a testing platform that incorporates the present invention are 
possible, with different modular organizations, control structures, and data structures, 
and written in any of many different programming languages. The testing platform 
architecture described above is extremely flexible, allowing for derivation of classes 
corresponding to functional components in order to adapt the testing environment to 

20 new types of test routines, new or different types of communications interfaces, new 
or different types of result output sinks, including printers, display devices, databases, 
files, and other such data sinks, to new and different user I/O devices and paradigms, 
and to new and different computing resource environments, including hardware 
platforms and operating systems. Shielding of the test execution engine and test 

25 routines from hardware and software dependencies may also be accomplished in an 
almost limitless number of ways. 

The foregoing description, for purposes of explanation, used specific 
nomenclature to provide a thorough understanding of the invention. However, it will 
be apparent to one skilled in the art that the specific details are not required in order 

30 to practice the invention. The foregoing descriptions of specific embodiments of the 
present invention are presented for purpose of illustration and description. They are 
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not intended to be exhaustive or to limit the invention to the precise forms disclosed. 
Obviously many modifications and variations are possible in view of the above 
teachings. The embodiments are shown and described in order to best explain the 
principles of the invention and its practical applications, to thereby enable others 
5 skilled in the art to best utilize the invention and various embodiments with various 
modifications as are suited to the particular use contemplated. It is intended that the 
scope of the invention be defined by the following claims and their equivalents: 



